How do personality traits modulate real-world gaze behavior? Generated gaze data shows situation-dependent modulations.

It has both scientific and practical benefits to substantiate the theoretical prediction that personality (Big Five) traits systematically modulate gaze behavior in various real-world (working) situations. Nevertheless, previous methods that required controlled situations and large numbers of participants failed to incorporate real-world personality modulation analysis. One cause of this research gap is the mixed effects of individual attributes (e.g., the accumulated attributes of age, gender, and degree of measurement noise) and personality traits in gaze data. Previous studies may have used larger sample sizes to average out the possible concentration of specific individual attributes in some personality traits, and may have imposed control situations to prevent unexpected interactions between these possibly biased individual attributes and complex, realistic situations. Therefore, we generated and analyzed real-world gaze behavior where the effects of personality traits are separated out from individual attributes. In Experiment 1, we successfully provided a methodology for generating such sensor data on head and eye movements for a small sample of participants who performed realistic nonsocial (data-entry) and social (conversation) work tasks (i.e., the first contribution). In Experiment 2, we evaluated the effectiveness of generated gaze behavior for real-world personality modulation analysis. We successfully showed how openness systematically modulates the autocorrelation coefficients of sensor data, reflecting the period of head and eye movements in data-entry and conversation tasks (i.e., the second contribution). We found different openness modulations in the autocorrelation coefficients from the generated sensor data of the two tasks. These modulations could not be detected using real sensor data because of the contamination of individual attributes. In conclusion, our method is a potentially powerful tool for understanding theoretically expected, systematic situation-specific personality modulation of real-world gaze behavior.

Difficulty with the preceding visual search affects brain activity in the following resting period.

It has been well-documented that brain regions related to a task are activated during the task performance. We investigated whether brain activity and functional connectivity during the rest period are affected by the preceding task. Participants performed visual search tasks with three search conditions, which were followed by a rest period. During the rest period, participants were asked to look at the display that did not show any visual stimuli. In the result, brain activity in occipital and superior parietal regions would be deactivated by the preceding task during the rest period after visual search tasks. However, the activity of the inferior frontal gyrus during the rest period, which is also part of the attention network, was not affected by the brain activity during the preceding visual search task. We proposed a new model for explaining how the cognitive demands of the preceding visual search task regulate the attention network during the rest period after the task. In this model, the cognitive demand changes with task difficulty, which affects the brain activity even after removing the visual search task in the rest phase.

Non-goal-driven eye movement after visual search task.

We investigated the functions and mechanisms of non-goal-driven eye movements, which are defined as eye movements induced when looking at visual stimuli on a display without engaging in a specific task or looking at a display without any visual stimuli or tasks. In our experiment, participants were asked to perform a visual search task on a display, which was followed by a rest period in which stimuli remained on the display or all stimuli were erased. During the rest period, the participants were asked to only look at the displays without engaging in any visual or cognitive tasks. We mainly analyzed the gaze-shift patterns in both task and rest periods, in which eye movements were classified in accordance with the angles of saccade directions in two consecutive saccades. The results indicate a significant difference between goal-driven eye movements, which were observed in the task period, and nongoal- driven eye movements, which were observed in the rest period. Scanning gaze-shift patterns dominated the task period, and backward and corrective-saccade-like gaze-shift patterns dominated the rest period. The gaze-shift pattern was affected by the task-difficulty during the task period. From these findings, we propose a model describing the oculomotor system in terms of goal-driven and non-goal-driven eye movements. In this model, the engagement levels of top-down and bottom-up control change along with task difficulty and are affected by the gaze-shift patterns during a visual search task. Decoupling of top-down control from the oculomotor system during a rest period induces backward saccades, resulting in fixation around the central part of a display. Therefore, we suggest that non-goaldriven eye movements play a crucial role in maintaining the readiness of the oculomotor system for the next search task.

Pupillary fluctuation amplitude preceding target presentation is linked to the variable foreperiod effect on reaction time in Psychomotor Vigilance Tasks.

Understanding temporally attention fluctuations can benefit scientific knowledge and real-life applications. Temporal attention studies have typically used the reaction time (RT), which can be measured only after a target presentation, as an index of attention level. We have proposed the Micro-Pupillary Unrest Index (M-PUI) based on pupillary fluctuation amplitude to estimate RT before the target presentation. However, the kind of temporal attention effects that the M-PUI reflects remains unclear. We examined if the M-PUI shows two types of temporal attention effects initially reported for RTs in the variable foreperiod tasks: the variable foreperiod effect (FP effect) and the sequential effect (SE effect). The FP effect refers to a decrease in the RT due to an increase in the foreperiod of the current trial, whereas the SE effect refers to an increase in the RT in the early part of the foreperiod of the current trial due to an increase in the foreperiod of the previous trial. We used a simple reaction task with the medium-term variable foreperiods (Psychomotor Vigilance Task) and found that the M-PUI primarily reflects the FP effect. Inter-individual analyses showed that the FP effect on the M-PUI, unlike other eye movement indices, is correlated with the FP effect on RT. These results suggest that the M-PUI is a potentially powerful tool for investigating temporal attention fluctuations for a partly unpredictable target.

Task-irrelevant object response to action enhances the sense of agency for controlling the object in automation.

The sense of agency (SoA) refers to the experience of controlling our bodies and tools. Recent automated systems require the operators to have less manual control, which decreases the SoA. This study investigated how to increase the SoA when operating automated systems, by focusing on the effect of an object's responses to operators' actions on the SoA. Participants applied brakes to a moving black circle by pressing a key, in order to stop the circle near a goal. Then, they estimated their SoA for stopping the circle. We informed them that there were automatic control trials in which the circle stopped independently of their keypress (86% of the trials). The circle's color briefly changed to white (i.e., flashed) when they pressed the key in a half of the automatic control trials. The SoA was higher with the flash than without it. However, the SoA neither increased when the circle flashed independently of a keypress nor when another object flashed due to a keypress. Furthermore, the keypress contingent object-flash did not influence the SoA when the participants controlled the circle manually. These results indicated that spatiotemporally contingent object responses to actions can enhance the SoA in automatic control situations.

How are emotional facial expressions detected rapidly and accurately? A diffusion model analysis.

Previous psychological studies have shown that people detect emotional facial expressions more rapidly and accurately than neutral facial expressions. However, the cognitive mechanisms underlying the efficient detection of emotional facial expressions remain unclear. To investigate this issue, we used diffusion model analyses to estimate the cognitive parameters of a visual search task in which participants detected faces with normal expressions of anger and happiness and their anti-expressions within a crowd of neutral faces. The anti-expressions were artificially created to control the visual changes of facial features but were usually recognized as emotionally neutral. We tested the hypothesis that the emotional significance of the target's facial expressions modulated the non-decisional time and the drift rate. We also conducted an exploratory investigation of the effect of facial expressions on threshold separation. The results showed that the non-decisional time was shorter, and the drift rate was larger for targets with normal expressions than with anti-expressions. Subjective emotional arousal ratings of facial targets were negatively related to the non-decisional time and positively associated with the drift rate. In addition, the threshold separation was larger for normal expressions than for anti-expressions and positively associated with arousal ratings for facial targets. These results suggest that the efficient detection of emotional facial expressions is accomplished via the faster and more cautious accumulation of emotional information of facial expressions which is initiated more rapidly by enhanced attentional allocation.

Model-Based Prediction of Operation Consequences When Driving a Car to Compensate for a Partially Restricted Visual Field by A-Pillars.

The partial restriction of a driver's visual field by the physical structure of the car (e.g., the A-pillar) can lead to unsafe situations where steering performance is degraded. Drivers require both environmental information and visual feedback regarding operation consequences. When driving with a partially restricted visual field, and thus restricted visual feedback, drivers may predict operation consequences using a previously acquired internal model of a car. To investigate this hypothesis, we conducted a tracking and driving task in which visual information was restricted to varying degrees. In the tracking task, participants tracked a moving target on a computer screen with visible and invisible cursors. In the driving task, they drove a real car with or without the ability to see the distant parts of a visual field. Consequently, we found that the decrease in tracking performance induced by visual feedback restriction predicted the decrease in steering smoothness induced by visual field restriction, suggesting that model-based prediction was used in both tasks. These findings indicate that laboratory-based task performance can be used to identify drivers with low model-based prediction ability whose driving behavior is less optimal in restricted vision scenarios, even before they obtain a driver's license. However, further studies are required to examine the underlying neural mechanisms and to establish the generalizability of these findings to more realistic settings.

Pupillary fluctuation amplitude before target presentation reflects short-term vigilance level in Psychomotor Vigilance Tasks.

Our daily activities require vigilance. Therefore, it is useful to externally monitor and predict our vigilance level using a straightforward method. It is known that the vigilance level is linked to pupillary fluctuations via Locus Coeruleus and Norepinephrine (LC-NE) system. However, previous methods of estimating long-term vigilance require monitoring pupillary fluctuations at rest over a long period. We developed a method of predicting the short-term vigilance level by monitoring pupillary fluctuation for a shorter period consisting of several seconds. The LC activity also fluctuates at a timescale of seconds. Therefore, we hypothesized that the short-term vigilance level could be estimated using pupillary fluctuations in a short period and quantified their amplitude as the Micro-Pupillary Unrest Index (M-PUI). We found an intra-individual trial-by-trial positive correlation between Reaction Time (RT) reflecting the short-term vigilance level and M-PUI in the period immediately before the target onset in a Psychomotor Vigilance Task (PVT). This relationship was most evident when the fluctuation was smoothed by a Hanning window of approximately 50 to 100 ms (including cases of down-sampled data at 100 and 50 Hz), and M-PUI was calculated in the period up to one or two seconds before the target onset. These results suggest that M-PUI can monitor and predict fluctuating levels of vigilance. M-PUI is also useful for examining pupillary fluctuations in a short period for elucidating the psychophysiological mechanisms of short-term vigilance.

Influence of levels of automation on the sense of agency during continuous action.

Recent advances in automation technology can lead to unsafe situations where operators lose their sense of agency over the automated equipment. On the other hand, increasing evidence has shown that providing operators with opportunities of continuous operation and helping them improve their performance on tasks through automation can boost their sense of agency. However, it is challenging to ensure that the operator maintains a sense of agency when working with a fully automated tool that removes him/her from the control loop. By demonstrating a tracking task in which participants continuously tracked a moving target through a cursor controlled by a joystick under different levels of automation, we illustrate how the participants' sense of agency and tracking performance were altered in accordance with the level of automation. The results showed that their sense of agency was enhanced by increasing automation but began to decline when the level of automation exceeded 90%. More generally, this suggests that allowing operators a little contribution to control over the continuous operation of an automated tool may be sufficient to maintain their sense of agency while yielding the maximum improvement in performance.

Putative ratios of facial attractiveness in a deep neural network.

Empirical evidence has shown that there is an ideal arrangement of facial features (ideal ratios) that can optimize the attractiveness of a person's face. These putative ratios define facial attractiveness in terms of spatial relations and provide important rules for measuring the attractiveness of a face. In this paper, we show that a deep neural network (DNN) model can learn putative ratios from face images based only on categorical annotation when no annotated facial features for attractiveness are explicitly given. To this end, we conducted three experiments. In Experiment 1, we trained a DNN model to recognize the attractiveness (female/male × high/low attractiveness) of face in the images using four category-specific neurons (CSNs). In Experiment 2, face-like images were generated by reversing the DNN model (e.g., deconvolution). These images depict the intuitive attributes encoded in CSNs of the four categories of facial attractiveness and reveal certain consistencies with reported evidence on the putative ratios. In Experiment 3, simulated psychophysical experiments on face images with varying putative ratios reveal changes in the activity of the CSNs that are remarkably similar to those of human judgements reported in a previous study. These results show that the trained DNN model can learn putative ratios as key features for the representation of facial attractiveness. This finding advances our understanding of facial attractiveness via DNN-based perspective approaches.

Neuroplastic Reorganization Induced by Sensory Augmentation for Self-Localization During Locomotion.

Sensory skills can be augmented through training and technological support. This process is underpinned by neural plasticity in the brain. We previously demonstrated that auditory-based sensory augmentation can be used to assist self-localization during locomotion. However, the neural mechanisms underlying this phenomenon remain unclear. Here, by using functional magnetic resonance imaging, we aimed to identify the neuroplastic reorganization induced by sensory augmentation training for self-localization during locomotion. We compared activation in response to auditory cues for self-localization before, the day after, and 1 month after 8 days of sensory augmentation training in a simulated driving environment. Self-localization accuracy improved after sensory augmentation training, compared with the control (normal driving) condition; importantly, sensory augmentation training resulted in auditory responses not only in temporal auditory areas but also in higher-order somatosensory areas extending to the supramarginal gyrus and the parietal operculum. This sensory reorganization had disappeared by 1 month after the end of the training. These results suggest that the use of auditory cues for self-localization during locomotion relies on multimodality in higher-order somatosensory areas, despite substantial evidence that information for self-localization during driving is estimated from visual cues on the proximal part of the road. Our findings imply that the involvement of higher-order somatosensory, rather than visual, areas is crucial for acquiring augmented sensory skills for self-localization during locomotion.

Explicit Sense of Agency in an Automatic Control Situation: Effects of Goal-Directed Action and the Gradual Emergence of Outcome.

Sense of agency (SoA), or the subjective feeling that "I am the agent controlling the object," is essential for learning and enjoying object manipulation. Recently developed automatic control systems, such as the cruise control systems in autonomous vehicles, require less manual control from the manipulators. It has to date been impossible to completely relieve operators of the need for manual control in many automatic control systems developed for tool-using situations. Therefore, it is important to examine how to maintain SoA (illusorily) during an automatic control situation. We investigated the effects of two typical characteristics of everyday tool-use situations on SoA when braking a moving object with a keypress. These characteristics included the presence of a goal (e.g., in driving situations, the driver steps on the brake pedal to stop the car at an expected position) and the gradual emergence of the outcome (e.g., the driver steps on the brake pedal and the car usually slows down first and then stops). We conducted an experiment in which participants stopped a moving object and then rated their SoA for stopping the object. Participants were explicitly informed that the object would sometimes stop independently of their keypress. Results showed that both characteristics decreased SoA in the manual control situation but increased SoA in the automatic control situation. Thus, these characteristics could be useful for maintaining a sense of agency in automatic control situations.

Primary Cognitive Factors Impaired after Glioma Surgery and Associated Brain Regions.

Previous studies have shown that cognitive impairments in patients with brain tumors are not severe. However, to preserve the postsurgical QOL of patients with brain tumors, it is important to identify "primary" cognitive functions and associated brain regions that are more vulnerable to cognitive impairments following surgery. The objective of this study was to investigate primary cognitive factors affecting not only simple cognitive tasks but also several other cognitive tasks and associated brain regions. Patients with glioma in the left (n = 33) and the right (n = 21) hemisphere participated in the study. Seven neuropsychological tasks from five cognitive domains were conducted pre- and 6 months postoperation. Factor analyses were conducted to identify "primary" common cognitive functions affecting the task performance in left and right glioma groups. Next, lesion analyses were performed using voxel-based lesion-symptom mapping (VLSM) to identify critical brain regions related to impairments of the primary cognitive functions. Factor analysis revealed two primary cognitive components in each glioma group. The first cognitive component in the left glioma group affected the digit span forward and backward tasks and concept shifting and the letter-digit substitution tasks. VLSM analysis revealed significant regions from the posterior middle temporal gyri to the supramarginal gyrus. The second cognitive component affected verbal memory, and verbal fluency tasks and VLSM analysis indicated two different significant regions, the medial temporal regions and the middle temporal gyrus to the posterior parietal lobes. The first cognitive component in the right glioma group affected positive and negative factor loadings on the task, such that the positive cognitive component affected only the Stroop color-word task. VLSM related to deficits of the Stroop task revealed significant regions in the anterior medial frontal cortex. On the other hand, the negative component affected concept shifting, word fluency, and digit span forward tasks, and VLSM revealed significant regions in the right inferior frontal cortex. It is suggested that primary cognitive functions related to specific brain regions were possibly affected by glioma resection.

Artificial Optic Flow Guides Visual Attention in a Driving Scene.

OBJECTIVE: The objective of this study was to investigate whether an artificial optic flow created by dot motion guides attention in a driving scene. BACKGROUND: To achieve safe driving, it is essential to understand the characteristics of human visual information processing as well as to provide appropriate support for drivers. Past research has demonstrated that expanding optic flow guides visual attention to the focus of expansion. Optic flow is an attractive candidate for use as a cue to direct drivers' attention toward the significant information. The question addressed concerns whether an artificial optic flow can successfully guide attention even in a traffic situation involving the optic flow that naturally occurs while driving. METHOD: We developed a visual search paradigm embedded in a video of a driving scene. Participants first observed an optic flow motion pattern superimposed on the video for brief period; next, when the optic flow and video ceased, they searched a static display for a target among multiple distractors. RESULTS: The target detection was faster when a target's locus coincided with the implied focus of expansion from the preceding optic flow (vs. other loci). CONCLUSION: The artificial optic flow guides attention and facilitates searching objects at the focus of expansion even when the optic flow was superimposed on a driving scene. APPLICATION: Optic flow can be an effective cue for guiding drivers' attention in a traffic situation. This finding contributes to the understanding of visual attention in moving space and helps develop technology for traffic safety.

A Novel Approach to Sensorimotor Skill Acquisition Utilizing Sensory Substitution: A Driving Simulation Study.

The aim of this study is to demonstrate the potential of sensory substitution/augmentation (SS/A) techniques for driver assistance systems in a simulated driving environment. Using a group-comparison design, we examined lane-keeping skill acquisition in a driving simulator that can provide information regarding vehicle lateral position by changing the binaural balance of auditory white noise delivered to the driver. Consequently, lane-keeping accuracy was significantly degraded when the lower visual scene (proximal part of the road) was occluded, suggesting it conveyed critical visual information necessary for lane keeping. After 40 minutes of training with auditory cueing of vehicle lateral position, lane-keeping accuracy returned to the baseline (normal driving) level. This indicates that auditory cueing can compensate for the loss of visual information. Taken together, our data suggest that auditory cueing of vehicle lateral position is sufficient for lane-keeping skill acquisition and that SS/A techniques can potentially be used for the development of driver assistance systems, particularly for situations where immediate time-sensitive actions are required in response to rapidly changing sensory information. Although this study is the first to apply SS/A techniques to driver assistance, further studies are however required to establish the generalizability of the findings to real-world settings.

Cerebellar activation associated with model-based estimation of tool-use consequences.

BACKGROUND: Dexterous tool use is considered to be underpinned by model-based control relying on acquired internal models of tools. In particular, this is the case in situations where available sensory feedback regarding the consequences of tool use is restricted. In the present study, we conducted an fMRI study to identify cerebellar involvement in model-based estimation of tool-use consequences using tracking tasks with different levels of visual feedback. METHODS: Twenty healthy right-handed adults participated in this study. These participants tracked a moving target on a screen with a cursor controlled by a joystick using their right hand during fMRI scanning. For each trial, the level of visual feedback for cursor position was randomly selected from three task conditions, namely, Precise, Obscure, and No conditions. RESULTS: A conjunction analysis across all task conditions found extensive activation of the right cerebellum, covering the anterior lobe (lobule V) and inferior posterior lobe (lobule VIII). Also, contrasts among the three task conditions revealed additional significant activation of the left superior posterior lobe (Crus I) in the No compared to the Precise condition. Furthermore, a post hoc psychophysiological interaction analysis revealed conditional modulation of functional coupling between the right, but not the left, cerebellar region and right frontoparietal regions that are involved in self-body perception. CONCLUSIONS: Our data show that the left Crus I is the only region that was more active in a condition where no visual feedback for cursor position was available. This suggests that the left Crus I region plays a role in model-based estimation of tool-use consequences based on an acquired internal model of tools.

Task-irrelevant optic flow guides attention in visual search.

Motion is an important factor in visual information processing. Studies have shown that global optic flow guides attention, but it remains unclear whether this attentional guidance occurs regardless of top-down attentional control settings for another endogenous cue. To address this issue, we developed a visual search paradigm in which a task-irrelevant optic flow starts and stops prior to a visual search task itself. Participants first observed an initial optic flow motion pattern for a brief period; next, they searched a static display for a target amongst multiple distractors. Results showed faster target detection when a target's locus coincided with the implied focus of expansion (FOE) from the preceding optic flow (vs. other loci). Eye-movement analyses revealed that initial saccades were drawn to the FOE during optic flow exposures and that relatively few saccades were needed to find targets contingent to the preceding FOE. The advantage of FOE for finding target occurred even when a salient feature singleton captured attention or when a task-relevant feature singleton was prioritized. Results of six experiments suggest that attentional control settings for a feature singleton do not over-ride a sustained influence of optic flow on attentional guidance.

Automatic detection of mind wandering in a simulated driving task with behavioral measures.

Mind wandering (MW) is extremely common during driving and is often accompanied by performance losses. This study investigated the use of driving behavior measurements to automatically detect mind wandering state in the driving task. In the experiment, participants (N = 40) performed a car-following task in a driving simulator and reported, upon hearing a tone, whether they were experiencing mind wandering or not. Supervised machine learning techniques were applied to classify MW-absent versus MW-present state, using both driver-independent and driver-dependent modeling methods. In the driver-independent modeling, we separately built models for participants with high or low MW and participants with medium MW. The optimal models can not offer a significant improvement than other models. So building effective driver-independent models with the leave-one-participant-out cross-validation method is challenging. In the driver-dependent modeling, we built models for each participant with medium MW. The best models of some participants were effective. The results indicate the development of mind wandering detecting system should take into account both inter-individual and intra-individual difference. This study provides a step toward minimizing the negative impacts of mindless driving and should benefit other fields of psychological research.

Peripersonal versus extrapersonal visual scene information for egocentric direction and position perception.

When perceiving the visual environment, people simultaneously perceive their own direction and position in the environment (i.e., egocentric spatial perception). This study investigated what visual information in a scene is necessary for egocentric spatial perceptions. In two perception tasks (the egocentric direction and position perception tasks), observers viewed two static road images presented sequentially. In Experiment 1, the critical manipulation involved an occluded region in the road image, an extrapersonal region (far-occlusion) and a peripersonal region (near-occlusion). Egocentric direction perception was worse in the far-occlusion condition than in the no-occlusion condition, and egocentric position perceptions were worse in the far- and near-occlusion conditions than in the no-occlusion condition. In Experiment 2, we conducted the same tasks manipulating the observers' gaze location in a scene-an extrapersonal region (far-gaze), a peripersonal region (near-gaze) and the intermediate region between the former two (middle-gaze). Egocentric direction perception performance was the best in the far-gaze condition, and egocentric position perception performances were not different among gaze location conditions. These results suggest that egocentric direction perception is based on fine visual information about the extrapersonal region in a road landscape, and egocentric position perception is based on information about the entire visual scene.